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A Motion Detecting Web Camera System 

Related Application 

This application is related to Application No. entitled, "A Web 

Microphone System", filed concurrently herewith, and Application No. 09/124,179 
5 entitled, "Digital Opaque Projector for Easy Creation and Capture of Presentation 
Material", filed July 28, 1998, which applications are assigned to the assignee of 
the present application. 

Field Of The Invention 

The present invention relates generally to the field of video capture; more 
10 particularly, to video camera systems having information processing capabilities 
for uploading pictures to a web server. 

Background of the Invention 

A web camera (i.e., "webcam") system consists of a video camera plus 
15 software that runs on a personal computer to periodically upload an image from 
the camera to a web page. The basic purpose of a web camera system is to 
post a reasonably live picture on a user-specified web page. Many webcam 
systems upload images on a periodic basis; for example, uploading an image 
once per hour. 

20 Figure 1 illustrates a web camera system 10, which is typical of the prior 

art. System 10 includes a video camera 1 1 that outputs a captured video image 
to a personal computer (PC) 12. Software running on PC 12 functions to 
periodically upload the captured video image to an Internet web page (i.e., a web 
server) shown in Figure 1 by block 13. Internet service providers (ISPs) 

25 commonly provide their patrons with a certain allocation of web page space for 
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personal use. This allows the user to upload images onto their web page 
periodically; with the frequency of uploading being dependent on the particular 
type of connection offered by the ISP. 

Presently, there are two shareware products in existence that relate to 
web cameras: Ispy ™ and Webcam32™. The Ispy webcam software functions 
to grab video images, save them as JPEG files, and then send the saved images 
automatically to a user-specified home page via the connection provided by the 
users' ISP. Ispy runs under Windows™ 95, Windows 98 and Windows NT 4.0; it 
also works with any video for Windows-compatible cameras and frame grabbers. 
Webcam32 is a Windows 95, Windows 98 and Windows NT application that 
allows video camera images to be displayed within a web page. Like Ispy, 
Webcam32 software is able to upload images to a web server to allow images to 
be obtained directly from the page. Both of these products include various 
simple image-processing features such as captioning of photos, day/time 
stamping, and text additions. 

Webcam32™ software also offers rudimentary motion detection, which is 
of primary use in security surveillance applications. For example, the 
Webcam32™ software allows images to be uploaded when, say, 25% of the 
pixels in the image frame change from one image frame to the next. Although 
this motion-detecting feature of the software product is useful in limited types of 
motion detection applications (e.g., security surveillance), it is not useful for 
different applications. For example, if the web camera system is intended for use 
in observing and recording wildlife activity, then this type of rudimentary motion 
detection does not work well. 

Another problem with today's webcam systems is the conflict between the 
desire to minimize the number of times the web camera contacts the ISP and the 
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need to capture "interesting" pictures (i.e., those containing certain kinds of 
motion). Most security surveillance type of web camera systems have a low 
threshold that results in the taking of many pictures whenever activity is detected. 
Uploading many pictures onto a web page presents a serious bandwidth 
problem. 

Furthermore, existing products such as Ispy and Webcam32 only provide 
the ability to capture images on a given schedule, e.g., once per hour, or 
whenever motion occurs, regardless of how often. If set to capture on a 
predetermined schedule, images that may be of interest to the user may end up 
being ignored. On the other hand, if the software is set to upload a video image 
whenever motion is detected, scenes containing frequent motion can tie up the 
user's phone lines. 

Thus, there is a need for a web camera system that overcomes the 
problems inherent in the prior art. 
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Summary of the Invention 



The present invention is a camera system for connection to a web server. 
The system comprises a video camera and a processor that periodically uploads 
images captured by the video camera in accordance with one of a plurality of 
motion detection algorithms. A first motion detection algorithm captures a current 
image frame when a pixel comparison between successive image frames 
exceeds a predetermined threshold. 
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Brief Description of the Drawings 

The present invention will be understood more fully from the detailed 
description which follows and from the accompanying drawings, which however, 
should not be taken to limit the invention to the specific embodiments shown, but 
5 are for explanation and understanding only. 

Figure 1 is a diagram of a prior art video capture system. 

Figure 2 is an example of an application of the present invention. 

Figure 3 is a conceptual diagram of various motion-detecting algorithms 
10 utilized in accordance with one embodiment of the present invention. 

Figure 4 is a flowchart illustrating one of the motion-detecting algorithms 
utilized in accordance with one embodiment of the present invention. 
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Detailed Description 

A web camera system that operates in accordance with one of a plurality 
of motion detection algorithms is described. In the following description, 
numerous specific details are set forth, such as specific operating modes, 
5 procedures, circuit elements, etc., in order to provide a thorough understanding 
of the invention. It will be obvious, however, to one skilled in the art, that these 
specific details may not be needed to practice the present invention. 

The state-of-the-art of web camera systems is such that there exists a 
conflict between the desire to minimize the number of times the web camera 

10 system dials up the Internet service provider and the desire to capture 

"interesting" pictures (e.g., those containing certain particular kinds of motion). 
As previously discussed, existing software products utilized to capture video 
images permit some rudimentary motion detection. These programs are utilized 
in applications concerned primarily with uploading images the instant motion is 

15 detected. While such programs are suitable for use in applications such as 
security surveillance systems, they suffer problems when used in different 
applications, e.g., observation of wildlife activity. 

Figure 2 illustrates three image frames 21-23 that may be captured 
utilizing a web camera system. In this case, the picture of interest is a bird 

20 feeding at a bird feeder station. The web camera is directed at the station, and 
operates to upload captured images either periodically, or in response to 
detected motion, or both. The problem that exists with prior art web camera 
systems that periodically upload images, say, every half-hour, is that a bird may 
arrive and leave many times within that time interval. In the event that a bird is 

25 not present at the scheduled image capture time, the only picture that will be 
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captured is one of an empty bird feeder. By way of example, this is the situation 
represented by image frame 21 . 

If a motion detecting feature is included in the web camera system, the 
taking of a picture may be triggered each time movement above a certain 
threshold is detected. In this case, when the number of pixels between two 
successive pictures changes (above a predetermined threshold) a new picture is 
captured and uploaded to the user's specified web site. By way of example, 
frame 22 of Figure 2 illustrates the arrival of a bird at the bird feeder, which 
triggers the video capture of an image frame. Note that many web camera 
systems in use today typically have low threshold settings that result in the taking 
of many pictures when the slightest activity is detected. This results in a 
bandwidth problem for the connection to the Internet service provider. 

Another concern relates to movement of the bird when it takes flight to 
leave the bird feeder station. Existing web camera systems with motion 
detecting features will trigger on this type of motion. Unfortunately, the last video 
image captured as a result of this type of motion is an empty bird feeder station, 
as represented by frame 23. In other words, the most recently captured image 
for uploading to the user's web site is that of the empty bird feeder, rather than 
the desired image of wildlife activity. 

The present invention solves the problem of motion detection and timed 
update by uploading one image each predetermined interval - selecting the best 
candidate video image that occurred during any given interval. The camera 
system includes a video camera coupled to a processor that operates in 
accordance with one of a plurality of motion detection algorithms to select an 
image for uploading to the user's web site. 
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The basic algorithm is as follows. Motion is first detected by performing a 
pixel comparison between successive image frames. When the pixel comparison 
between a current image frame and a previous image frame exceeds a 
predetermined threshold level, software running on the processor saves the 
5 current picture as a candidate for uploading. A simple frame grabber technology 
may be utilized for capture of the video image. 

The processor may be programmed to periodically upload the current or 
last candidate image. For example, a typical web camera system may operate 
by uploading the last candidate image once every hour. If no motion is detected 
10 in the past hour, there are two options: either no uploading of any image, or 
upload a current image captured by the camera regardless of whether motion is 
detected. 

Figure 3 illustrates a conceptual diagram of various detecting algorithms 
utilized in accordance with one embodiment of the present invention. Details of 
15 this embodiment are provided below and also in the attached Appendix, which 
contains a code listing of a software program that implements these algorithms. 

With reference to Figure 3, a camera outputs a color image, which is then 
converted to a black-and-white (B/W) pixel representation of a current image 
frame. Three different modes of motion detection are then provided to capture 

20 specific types of events. In the basic mode of motion detection, the web camera 
system selects the most recent image containing motion above a certain 
threshold. This image represents the candidate image for uploading to the user- 
specified web server. As previously discussed, the basic mode of motion 
detection involves a straightforward pixel-by-pixel comparison between a current 

25 frame and a previously captured image frame. 
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A second type of motion detection mode of operation is referred to as 
"stable-change" detection. The stable-change mode of motion detection is 
designed to capture persistent changes in a scene, while ignoring relatively 
simple motions. For example, the stable-change mode of operation is useful in 
5 video conferencing applications where the video camera is aimed at a desk or 
whiteboard where a speaker is placing written subject matter or other objects in 
front of the camera to facilitate discussion. In such applications, it is useful to 
capture the notes written on a board, or otherwise presented in front of the 
camera, while ignoring extraneous motion such as writing on the board, finger 
10 pointing, etc. 

The stable-change mode of operation is aimed at detecting stable 
changes to a particular scene and operates in accordance with an algorithm that 
captures an image frame a certain time period following a last detected motion 
event. The time duration may be programmable, timed, or fixed depending on 

15 what is being viewed and what is to be captured. In the video conferencing 
application discussed above, software running on the processor operates to 
capture a video image frame and copy or upload it to remote sites whenever a 
new writing (or other object for presentation) is placed in front of the video 
camera. The stable-change mode of operation ignores constant ongoing activity 

20 in the field of view. 

As can be seen in Figure 3, the stable-change algorithm compares a 
current frame against a last stable frame and selects as a candidate picture the 
last stable frame when no motion has been detected for a certain duration (e.g., 
two seconds). 

25 The third mode of motion detection operation is referred to in this 

application as "novel" motion detection. The novel motion detection mode of 
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operation solves the problem that arises in certain applications such as 
observation of wildlife activity wherein the motion of a bird arriving at a feeder is 
very similar to the motion of the bird departing from the feeder. A webcam 
system operating in accordance with only a basic motion detection algorithm -- 
5 which simply saves the most recent image with motion -- cannot distinguish 
between these two types of events. In other words, the basic motion detection 
algorithm captures the image of the just vacated bird feeder for uploading to the 
web server, rather than the picture of the bird that left, simply because it triggers 
on the last motion detected. 

10 In solving this problem, the novel motion detection algorithm compares an 

image that contains motion against the most recent stable image, as described 
above. Images that are not substantially different from the stable image frame 
are ignored. 

Figure 3 illustrates that novel motion detection involves not only detection 
15 of motion, based on a pixel comparison of a current frame against a previous 
frame, but also a comparison between the current frame and a last stable frame. 
For example, as applied to the empty bird feeder problem, the last stable frame is 
that of the recently vacated bird feeder. According to this third algorithm, motion 
is recorded instead of the absence of motion. That is, the detection of motion, 
20 based on a pixel comparison between the current frame and previous frame, 
triggers capture of a current image frame in a buffer. The buffer may be any one 
of a variety of buffers, such as a circular buffer, with a capacity to store a 
sufficient number of image frames. 

In this mode, each time motion is detected, the current image is captured 
25 into a circular buffer. In other words, in the novel motion detection mode of 
operation, pictures are captured into a buffer at times other than the last stable 
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frame. The detection of a last stable frame triggers the uploading of an image 
from the circular buffer that was captured some predetermined time prior to the 
triggering event. In the bird feeder example, the uploaded image might be an 
image frame captured several seconds prior to the last detected motion, e.g., an 
image of the bird prior to leaving. 

In one embodiment of the present invention the selected image represents 
a candidate picture that may be uploaded to a web server at a predetermined 
interval. Generally speaking, the user may set the interval for uploading the last 
candidate picture, as well as the particular mode of motion detection to be 
utilized. 

Figure 4 is a flow chart illustrating the novel motion detection algorithm in 
accordance with one embodiment of the present invention. The flow chart begins 
at step 31 , which indicates the capture of a current image frame by the video 
camera. If a colored video camera is utilized, the color image may be 
transformed to a black and white pixel representation. At step 33, a pixel 
comparison is made between the current frame and a previously captured frame. 
If the pixel comparison indicates that the number of pixels between the two 
frames exceeds a predetermined threshold, then the algorithm proceeds to step 
34. If the predetermined threshold is not exceeded (i.e., no motion is detected) 
the flow chart returns to step 31 . 

When basic motion is detected, the pixel comparison causes a motion 
signal to be asserted by logic circuitry in the processor. This is illustrated in 
Figure 4 by step 34. Assertion of the motion signal causes the current image to 
be loaded into a candidate buffer which holds the most recent images for periodic 
uploading to a web site. 
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Both the current image frame and the previous image frame may be held 
in separate buffers after being captured by the video camera. In this particular 
mode of operation, a circular buffer may be utilized as a candidate buffer for 
holding the most recent images captured responsive to the motion signal. The 
5 storing of the image frames in the circular buffer is represented in Figure 4 by 
step 35. 

Step 36 is a determination of whether a stable image frame is detected. If 
not, the algorithm returns to the beginning step 31 . On the other hand, if a stable 
frame is detected, then one of the stored frames is selected for uploading to a 
10 web server. The particular frame that may be chosen is the frame that occurred 
a predetermined time prior to the detection of a stable frame. For example, in the 
bird feeder example, it is useful to select an image that was captured several 
seconds prior to the last detected motion of the bird leaving the feeder. 
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I claim: 



Claims 



1 1 . A camera system for connection to a web server comprising: 

2 a video camera; 

3 a processor that periodically uploads images captured by the video 

4 camera in accordance with one of a plurality of motion detection algorithms, a 

5 first motion detection algorithm capturing a current image frame when a pixel 

6 comparison between successive image frames exceeds a predetermined 

7 threshold. 

1 2. The camera system of claim 1 wherein the processor uploads the 

2 current image frame at programmedjptelvals. 

1 3. The camera system of claim 1 wherein the plurality of motion detection 

2 algorithms further comprises a second motion detection algorithm that captures a 

3 stable frame after a certain duratiortlia^lapsed since the predetermined 

4 threshold has been exceeded. 

1 4. The camera system of claim 1 wherein the plurality of motion detection 

2 algorithms further comprises a third motior> detection algorithm that captures a 

3 recent motion frame that occurs a predetermined time period prior to the 

4 occurrence of a stable frame, the stable frame occurring after a certain duration 

5 has elapsed since the predetermined threshold has been exceeded. 

1 5. The camera system of claimJS wherein the plurality of motion detection 

2 algorithms further comprises a third motion detection algorithm that captures a 
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3 recent motion frame that occurs a predetermined time period prior to the 

4 occurrence of the stable frame. 

1 6. The camera system of claim 4 wherein the processor includes a circular 

2 buffer to successively store motion captured in image frames in which the 

3 predetermined threshold is exceeded. 

1 7. A web camera system for uploading pictures to a web site comprising: 

2 a video camera; 

3 a processor coupled to the video camera, the processor including: 

4 a current frame buffer to hold a current image captured by the video 

5 camera; 

6 a previous frame buffer to hold a previous image captured prior to 

7 the current image; 

8 a candidate buffer to hold a most recent image for periodic 

9 uploading to the web site; 

10 logic circuitry to perform a pixel comparison between the current 

11 image and the previous image, the logic circuitry asserting a motion signal when 

12 the pixel comparison exceeds a predetermined threshold; 

13 the processor operating according to one of a plurality of modes, in a first 

14 mode of operation the current image is loaded into the candidate buffer 

15 responsive to the motion signal. 

1 8. The web camera system of claim 7 wherein in a second mode of 

2 operation the candidate buffer is loade<fwith the current image after a certain 

3 duration has elapsed following assertion of the motion signal. 



042390. P7389 



-14- 



1 9. The web camera system of claim 8 further comprising: 

2 a circular buffer to store successive current images when the motion 

3 signal is asserted; 

4 in a third mode of operation the processor selecting one of the current 

5 images stored in the circular buffer for loading into the candidate buffer once the 

6 motion signal has been de-asserted for a predetermined time. 

1 1 0. The web camera system of claim 7 further comprising: 

2 a circular buffer to store successiv^urrent images when the motion 

3 signal is asserted; 

4 in a third mode of operation the processor selecting one of the current 

5 images stored in the circular buffer for loading into the candidate buffer once the 

6 motion signal has been de-asserted for a predetermined time. 

1 1 1 . A method of operating a web camera system comprising: 

2 capturing a current image frame from a video camera; 

3 asserting a motion detection signal when a pixel comparison between the 

4 current image and a previous image frame exceeds a predetermined threshold; 

5 storing in a buffer successive image frames captured from the video 

6 camera while the motion detection signal is asserted; 

7 de-asserting the motion detection signal when the predetermined 

8 threshold is no longer exceeded for the current image frame; 

9 selecting from the buffer a certain one of the successive image frames as 

10 a candidate picture once the motion detection signal has been de-asserted for a 

11 certain duration; and 

12 uploading the candidate picture to a web site. 
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1 1 2. The method according to claim 1 1 wherein the buffer is a circular buffer 

2 having a capacity to store a plurality of image frames. 

1 13. The method according to claim 1 1 wherein the uploading step is 

2 performed at periodic time intervals. 

1 14. The method according to claim 1 1 wherein the certain one of the 

2 successive image frames is stored a predetermined time before a last image 

3 frame is stored in the buffer prior to de-assertion of the motion detection signal. 

1 1 5. A method of operating a web camera system comprising: 

2 capturing a current image frame from a video camera; 

3 asserting a motion detection signal when a pixel comparison between the 

4 current image and a previous image frame exceeds a predetermined threshold; 

5 storing in a buffer successive image frames captured from the video 

6 camera while the motion detection signal is asserted; 

7 de-asserting the motion detection signal when the predetermined 

8 threshold is no longer exceeded for the current image frame; 

9 selecting as a candidate picture either: 

10 (i) the current image when the motion detection signal is 

11 asserted; 

12 (i) the current image a first duration following de-assertion of 

13 the motion detection signal; or 

14 (ii) a certain one of the successive image frames from the 

15 buffer once the motion detection signal has been de- 

16 asserted for a second duration; and 
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17 



uploading the candidate picture to a web site. 



1 16. The method according to claim 1 1 wherein the buffer is a circular buffer 

2 having a capacity to store a plurality of image frames. 

1 17. The method according to claim 1 1 wherein the uploading step is 

2 performed at periodic time intervals. 

1 18. The method according to claim 1 1 wherein the certain one of the 

2 successive image frames is stored a predetermined time before a last image 

3 frame is stored in the buffer prior to de-assertion of the motion detection signal. 
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Abstract of the Disclosure 

A camera system that includes a video camera and a processor, which 
periodically uploads images captured by the video camera to a web server in 
accordance with one of a plurality of motion detection algorithms. A first motion 
5 detection algorithm captures a current image frame when a pixel comparison 
between successive image frames exceeds a predetermined threshold. 
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36 
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37 SELECT ONE OF THE 
STORED FRAMES FOR 
UPLOADING TO WEB SERVER 



iq, 4 



// Called when an ID_C0NTR0L_NEW_FRAME message is received 
// as a result of the frame callback getting a new frame, 
void CSnapshotCtrl: : OnControlNewf rame ( ) 
{ 



// time (ras since bootup) when we started processing this frame. 

// time (ms) elapsed since the previous frame was snapped. 

// "send this frame as a snapshot" 

// "motion occurred between the previous frame and the current frame. 1 

// "motion occurred between the stable frame and the current frame." 

// "a stable change happened" 

// time (ms) elapsed since movement was detected. 

// the absolute threshold to use in the difference calculation. 

// the motion rectangle, in video coordinates. 

// number of pixels that have changed (vs. moved), in percent. 

// the time the frame was taken, in seconds since the epoch. 

// text label for the current picture. 



// region-of -interest for differences of the images (or NULL if none) 



DWORD startTime; 
DWORD endTime; 
DWORD msTimeSincePrev; 
BOOL sendFrame; 
BOOL motionOccurred; 
BOOL novel tyOccur red; 
BOOL changeOccurred; 
DWORD msldle; 
int dif fThreshold; 
CRect moveRect; 
double pcNumChanged; 
CTime CtNow; 
CString cStrTag; 
double fDiff; 
IPLStatus iplStat; 
IplROI *pRoi = NULL; 

IplROI *prevRoil, *prevRoi2 , *prevRoi3; // previous region-of -interest for images we mess with 



LPCSTR pMsg; 
; IplLUT *pLut [3] ; 
IplLUT lut [3] ; 
int lutKey[256 * 3] ; 
int lutValue[256 * 3]; 
int lutldx; 
int keyldx; 



// points to a static error message. 

// pointers to each lut [ ] entry. 

// the Lookup- table for each channel. 

// the key values for each channel's LUT. 

// the value values for each channel's LUT. 



sendFrame = FALSE; 
: motionOccurred = FALSE ; 

noveltyOccurred = FALSE; 
; changeOccurred = FALSE; 
I startTime = timeGetTime ( ) ; 
\ ctNow = CTime : : GetCurrentTime ( ) ; 

: // Keep our callback from overwriting our data. 

1 m__f rameBusy = TRUE ; 



//If the video was closed a while ago, ignore this message. 
// If we've already processed this frame, ignore it. 

// This helps when we are capturing frames faster than we can process them. 

if ( !m_pFrameCam || !m_plplln || im_readyFrameIn) { 
m_f rameBusy = FALSE; 
return; 

} 



// If the camera format is compressed, decompress it first so that IIPL can handle it. 
// Otherwise, just copy it. 



if (mJiicD) { 

if (ICERR_OK 1= ICDecompress (m_hicD, OL, 
&m_pFmtCam- >bmi Header , m_pFrameCam , 
&m_pFmtIn->bmiHeader , na__pFrameIn) ) { 
Stop() ; 

AfxMessageBox( "Frame decompress failed."); 
m_readyFrameIn = FALSE; 
m_f rameBusy - FALSE; 
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return; c 

i r 
} 

} else { 

if {m__pFmtCam->bmiHeader .biSizelmage != m_pFmtIn->bmiHeader . biSizelmage) { 
TRACE ( "OnControlNewFrame: camera imagesize ! = input imagesize\n" ) ; 

} 

memcpy (m_pFrameIn, m_pFrameCam, m_pFmt In- >bmiHeader . bi Size Image ) ; 

} 

// Convert the image to Intel Image Processing Library format for further processing. 

iplStat = iplConvertFromDIBSep(&m__pFmtIn->bmiHeader / (const char *) m_pFrameIn, m_plplln) ; 
if (iplStat != 0) { 
StopO; 

AfxMessageBox ("Couldn't convert camera output to RGB-24 format"); 
m„readyFrameIn = FALSE; 
m_f rameBusy - FALSE; 
return ; 

} 

// If the user wants rotate the image, do that. 

// (mirroring, on the other hand, is a preview function performed in OnDraw) . 
//A 180-degree rotation is equivalent to mirroring horizontally and vertically. 

r^-if (m_rotate) { 

ft iplMirror (m_plplln, m_plplln, -1); 
% > 

©// Figure out how much time has elapsed since the previous snapshot, 

jjl msTimeSincePrev = startTime - m_jnsPrevSnapTime; 

I II Calculate our absolute pixel difference threshold: 

™ // It's the per-pixel noise * 2 {because noise is additive) * 255 (because that's our luminanc 
e l|mge) 

Jf„ // divided by 100 because our noise is expressed in percent, 
II rounded to the nearest integer. 

m diffThreshold = (int) ( (m_f PixelNoise * 2.0 * 255.0) / 100.0 + 0.5); 

// Find our motion-detection rectangle in valid video coordinates. 

moveRect = m„rDetect; 
if (moveRect .left < 0) { 
moveRect . left = 0; 

} 

if (moveRect . top < 0) { 
moveRect . top = 0 ; 

} 

if (moveRect .right > m_plplln->width) { 
moveRect .right = m_plplln->width; 

} 

if (moveRect .bottom > m_plplln->height ) { 
moveRect .bottom = m_plplln->height ; 

} 

switch (m_snapMode) { 
case SNAP_BULB: 

// bulb mode is really a "don't take a picture unless forced" — do nothing, 
break; 
case SNAP„TIME_LAPSE : 

// Time lapse is really "try to send every frame, but all modes are throttled by mjmsTimeL 
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apse" _ < 

sendFrame = TRUE; 

break ; 
case SNAP_ON__MOTION: 
case SNAP_ON_NOVELTY : 
case SNAP_ON_CHANGE : 

// motion- and change-detection modes are handled below. 

break; 

de fault: // unknown snap mode. 
Stop { ) ; 

ASSERT (FALSE) ; 

} 

// If we're going to do any sort of motion/change-detection on this 
// do it. 



switch (m_snapMode) { 

case SNAP_BULB: 

case SNAP_TIME_LAPSE: 

// these modes aren't motion modes, 
break; 
case SNAP_0N_M0TION: 
case SNAP_ON_NOVELTY : 
incase SNAP_ON_CHANGE : 

fl[ // The motion-detection modes require limiting the detection to the motion rectangle. 
4f f / Create a region-of -interest for the motion rectangle. 

m pRoi = iplCreateROKO, moveRect . lef t , moveRect . top, moveRect .Width () , moveRect . Height ()) ; 

ffi if (IpRoi) { 

[|| Stop { ) ; , 

ff% AfxMessageBox ("Couldn't create region-of -interest for motion detection."); 

7 m_readyFrameIn = FALSE ; 

L,. m_f rameBusy = FALSE; 

"2 return; 

Si > 

h* // Convert the full-color image to an 8 -bit /pixel luminance- only image, 

if* // blur a bit first to lower the noise caused by pixel noise and camera vibration. 

2 //ZZZ the docs say iplFixedFilter supports in-place, but the routine errored when I tried 

to do that. 

iplColorToGray (m_plplln, m_pIplTmp) ; 

iplFixedFilter (m pIplTmp , m_pIp lMotion, IPL_GAUSSIAN_3x3 ) ; 

//ZZZ I want to add an edge-enhancement^ to avoid triggering on shadows or gross lighting c 

hanges , ^ 
//ZZZ but don't have time at the moment to build it. 

// If we're just starting, 

// copy this frame to the reference so we don't falsely tri&^er on motion. 

if ( !m__startFrameSeen) { 

iplCopy (m_pIplMotion, m__pIplMotionRef ) ; 

} 

// Find how different the current image, is from the motion-detection reference. 
// pixel„is_different - abs {a - b) > (pi,xel_noise + pixel_noi£e) . 

// This function essentially says "howVmkny corresponding pixels differ by greater than tn 
e noise in each pixel?" 
// 

// Since iplSubtract { ) performs saturation arithmetic, 
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//we Ijave tp perform two subtractions and sum their results 
7/ in order to implement our function. 

// Limit the region of interest to our motion-detection rectangle, 
// and limit the division to its number of pixels. 

//ZZZ can we just use iplSetRoi() on the images' ROI 1 s instead of the pointer-setting stuf 
//ZZZ the documentation really isn't explicit about how to set the ROI of an image. 

prevRoil = ra„pIplMotion->roi ; 
m_pIplMotion->roi = pRoi; 
prevRoi2 = m_pIplMotionRef ->roi ; 
m_pIplMotionRef->roi = pRoi; 
prevRoi3 = m_jpIplTmp->roi ; 
m_pIplTmp->roi = pRoi; 

iplSubtract (m_pIplMotion, m_pIplMotionRef , m_pIplTmp) ; 
iplThreshold(m_jpIplTmp, nupIplTmp, dif fThreshold) ; 
fDiff = iplNorm(mjIplTmp, NULL , IPL_L1) / 255.0; 

iplSubtract (m_pIplMotionRef , m_pIplMotion, m_pIplTmp) ; 
iplThreshold(m_pIplTmp, itupIplTmp, dif fThreshold) ; 
fDiff += iplNorm(m_pIplTmp, NULL t IPL_L1) / 255.0; 

// Restore the regions of interest. 

m_pIplMotion->roi = prevRoil; 
m_pIplMotionRef->roi = prevRoi2; 
m_pIplTmp->roi = prevRoi3; 

// fDiff is now the number of pixels that differed. 
// Convert it to the percent of pixels that differed. 

fDiff /= ((double) moveRect . Width ( ) * (double) moveRect . Height ()) ; 
m_fCurrentMotion = fDiff * 100.0; 

// If enough pixels have changed, call it motion. 
/ / Note when our most recent motion occurred 

// and that our change-detection algorithm doesn't have to wait for motion to occur any mo 

if (m„fCurrentMotion > itufMotionThreshold) { 
motionOccurred = TRUE; 
m_msPrevMoveTime = startTime; 
m_waitingForMovement - FALSE; 

} 

// Now that we're done with the previous edge-enhanced motion reference, 
// update it. 

iplCopy (m_jpIplMotion, m_pIplMotionRef ) ; 

switch (m_snapMode) { 
case SNAP_BULB: 
case SNAP_TIME_LAPSE : 
case SNAP_ON„MOTION: 

// these are not change-relative cases. 

break ; 

case SNAP_ON_NOVELTY : 
case SNAP_ON_CHANGE : 
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* , * /A Find .the number of pixels that Ijave changed 
* * if from our previous change-reference to our current frame, 

// using the same algorithm as above. 

// If we're just starting, 

// copy this frame to the reference so we don't falsely trigger on change. 

if ( !m_s tart Frame Seen) { 

iplCopy (irupIplMotion, m_pIplChangeRef ) ; 

} 

// Limit the region of interest to our motion-detection rectangle, 
// and limit the division to its number of pixels. 

prevRoil = m_pIplMotion->roi ; 
m„pIplMotion->roi = pRoi; 
prevRoi2 = m_pIplChangeRef->roi ; 
m_pIplChangeRef->roi = pRoi ; 
prevRoi3 = m_pIplTmp->roi; 
m__pIplTmp->roi = pRoi; 

iplSubtract (m„pIplMotion, m_pIplChangeRef , m_pIplTmp) ; 
iplThreshold(m_pIplTmp, m_pIplTmp, dif fThreshold) ; 
fn . fDiff = iplNorm(m_pIplTmp, NULL, IPL_L1) / 2 55.0; 

iplSubtract (m_pIplChangeRef , rtupIplMotion, m_pIplTmp) ; 
W iplThreshold(m^IplTmp, irupIplTmp, dif fThreshold) ; 

il fDiff += iplNom(m_pIplTmp / NULL , IPL_L1) / 255.0; 

ffl // Restore the regions of interest. 

ff, m_pIplMotion->roi = prevRoil; 

m_pIplChangeRef->roi = prevRoi2; 
L,, m_pIplTmp->roi = prevRoi3; 

tfl // fDiff is now the number of pixels that differed. 

ft! // Convert it to the percent of pixels that differed, expressed in lOths of a percent. 

yf| fDiff /= ({double) moveRect .Width ( ) * (double) moveRect . Height ()) ; 

pcNumChanged = fDiff * 100.0; 

//If enough pixels have changed, 

// this is novelty (change from the last stable change) . 

if (pcNumChanged > m_fMotionThreshold) { 
novel tyOccurred = TRUE; 

} 

// See how long it's been since motion happened. 

// If it's been long enough, and we aren't waiting for some motion to happen first, 
// we can see if a stable change has happened. 

msldle = startTime - m_msPrevMoveTime; 

if ( ( !m_waitingForMovement && msldle > m_msIdleThreshold) || m_forceSnap) { 

m_waitingForMovement = TRUE; // (because starting with the next frame, we're wa 

iting for new movement) 



ge. 



// If enough pixels have changed from the last stable scene, call it a stable chan 
// and record it as the reference for detecting future changes, 
if ( {pcNumChanged > m_fMotionThreshold) || m_forceSnap) { 
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, - changeOccurred = TRUE ; 

iplCopy (m_pIplMotion, m_pIplChangeRef ) ; 

} 

} 



break; 

default: // unknown snap mode. 
Stop ( ) ; 

ASSERT (FALSE) ; 



// Now we're done with the region of interest for motion-detection. 



if (SpRoi) { 
Stop{) ; 

ASSERT (FALSE) ; 

} 

iplDeleteROI (pRoi) ; 
pRoi = NULL; 



break; 

default: // unknown snap mode. 
Stop ( ) ; 
pr ASSERT (FALSE) ; 

J// Now that we know whether or not there's motion and change, 
""SV/ decide whether we need to send the image. 

Uj switch (m_snapMode) { 
fi'-case SNAPJBULB: 
pease SNAP_TIME„LAPSE: 

_ // These are non-motion modes, handled elsewhere. 

f-~ : break; 

^ case SNAP„ON__MOTI ON : 

If; // If motion happened, send the frame. 
y $ if (motionOccurred) { 
r 8 " sendFrame = TRUE; 

k/ 9 x break; 

case SNAP_ON_NOVELTY : 

// If motion occurred relative to the last stable change, 
// send the frame. 



if ( novel tyOccurred) { 
sendFrame = TRUE; 

} 

break; 
case SNAP_ON_CHANGE : 

// If the image is stable and different enough from the last stable change, 
// send the frame. 



if (changeOccurred) { 
sendFrame = TRUE; 

} 

break; 

default: // unknown snap mode. 
Stop ( ) ; 

ASSERT (FALSE) ; 



// If the user has asked for image histogram equalization, do it. 
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/J We* do this - AFTER all the raotion-detectiosi 

// because histogram equalization increases the pixel noise as the image gets more uniform, 
// ultimately converting a black image into full-brightness-range noise. 

if (m_doHistogramEQ) { 

// intialize the lookup tables for each channel. 

//ASSERT (m_plplln->numchannels == 3); 
for (lutldx - 0; lutldx < 3; ++lutldx) { 
pLut [lutldx] = &lut [lutldx] ; 

lut [lutldx] .num = 256; 

lut [lutldx] .key = &lutKey[256 * lutldx]; 
lut [lutldx] .value = &lutValue [256 * lutldx] ; 
lut [lutldx] .factor = NULL; 

lut [lutldx] . interpolateType = IPL_LUT_INTER; 

for (keyldx = 0; keyldx < 256; ++keyldx) { 
lutKey [lutldx * 256 + keyldx] - keyldx; 
lutValuef lutldx * 256 + keyldx] = 0; 

} 

} 

// Calculate the image histogram, 

// then equalize that histogram and apply it to the image. 

//ZZZ the pre-equalized histogram could be useful for detecting that, for example, 
//ZZZ the image is totally black and isn't worth sending. 

iplComputeHisto (m_plplln, &pLut [0] ) ; 
iplHistoSqualize (m_plplln, injplplln, &pLut[0]); 

} 

// If this frame is to be forced, send it. 

if (m_f orceSnap) { 

sendFrame = TRUE; 

} 

// If we've decided to send this frame, 

// create a fresh IPL copy of it and hand that copy off to the desired window. 

if (sendFrame) { 

Ipl Image *plpl = NULL; // the IPL image copy to send away. 

plpl = iplClonelmage (m_plplln) ; 
if (Iplpl) { 
Stop() ; 

AfxMessageBox ( "Couldn ' t create a duplicate of the captured image" ) ; 
m_readyFrameIn = FALSE; 
m_frameBusy = FALSE; 
return; 

} 

// Tag the copy with whatever text the sender wanted. 

// We don't tag the original because the tag would only flicker past in the preview. 

cStrTag = m_labelText; 

if (m_doLabelTime) { 

if ( IcStrTag.IsEmpty () ) { 
cStrTag += " " ; 



CString t = ctNow. Format ( "%I : %M%p" ) ; // (hnmin am/pm) 
t . MakeLower { ) ; 
cStrTag += t; 



if (m„doLabelDate) { 

if ( IcStrTag.IsEmpty () ) { 
cStrTag += " " ; 

} 

cStrTag += ctNow. Format ( "%B %d"); // (%B = full month name, %b = abbreviated month n 
ame, %d - day of month) 
} 



// If there's nothing to tag, don't tag the image. 



if ( IcStrTag.IsEmpty () ) { 

pMsg = Taglpllmage (plpl, (LPCSTR) cStrTag, m_labelColorBk, m_labelColorText ) ; 
if (pMsg) { 
Stop ( ) ; 

AfxMessageBox(pMsg) ; 

iplDeallocate {plpl , IPL_IMAGE_HEADER | IPL_IMAGE_DATA) ; 

plpl = NULL; 
==';. m__readyFrameIn = FALSE; 

m_frameBusy = FALSE; 
r\ return; 

-'i > 

> 

iy // Make this frame "pending", 

IV\ II If we can't send it right away, we'll send it when we can. 

[f;: // This arrangement makes motion- and change-detection modes be properly throttled by m_ms 
TimeLapse : 

^ // If motion or change occurred during the time-lapse interval, 
\S* // that moved or changed frame is sent once the interval expires. 

^' if (m„plpl Pending) { 

H' : iplDeallocate (m_jplp IP ending, IPL_IMAGE_HEADER | IPL_IMAGE_DATA) ; 

m_plpl Pending = NULL; 

it 3 

m__plp IP ending = plpl ; 

plpl - NULL; // (since we've moved it to Pending) 

} 



// If the time-lapse has expired si^ce we last sent a frame (or the user forced this frame) , 
// and we have a trame to senu, 

// send or write that pending trame and note the time we sent/wrote it. 



if (m_f orceSnap) { 

if ( !m__plp IP ending) { // (forcing a frame had better force the frame into pending state) 
Stop ( ) ; 

ASSERT (FALSE) ; 

} 

} 

if ( (m_f orceSnap || msTimeSincePrev >= m_msTimeLapse) && m__plp IP ending) { 
m_msPrevSnapTime = startTime; 



if ( 1 m_saveToName . I sEmpty ( ) ) { 

pMsg = Writelpl (m__plpl Pending, m_saveToName, m_jpegQuality) ; 
if (pMsg) { 
Stop ( ) ; 

AfxMessageBox (pMsg) ; 
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m - ^ . ' , iplDeallocate (m_plpl Pending, IPL_IMAG t E_HEADER j IPL_IMAGE_DATA) ; 

m_plpl Pending = NULL; 
m_readyFrameIn = FALSE; 
m_frameBusy ~ FALSE; 
return ; 

} 

} 

//m_pWndNotify->PostMessage (m„msgNotify , 0, (LPARAM) m_plpl Pending ) ; // (a SendMessage ( ) w 
ould confuse the threads} 

iplDeal locate (m_plpl Pending, IPL__IMAGE__HEADER | I PL_IMAGE_DATA ) ; 
m_plpl Pending = NULL; 

// Tell our container that an image file is ready for them. 
FirelmageReady ( ) ; 

} 

// Now we can forget whether the user forced this frame 
// and whether this was the first frame after a Start () . 

m_forceSnap = FALSE; 
m_startFrameSeen = TRUE; 

// Note how long it took to process this frame. 
O endTime = timeGetTime { ) ; 
if I m_frameTime = endTime - startTime; 

."li! // let our preview window know it's time to redraw. 
"™ InvalidateControl ( } ; 

y& m__readyFrameIn = FALSE; // {because we're done processing the current frame) 
yl m_frameBusy = FALSE; 
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Title 37, Code of Federal Regulations, Section 1.56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1.98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1 ) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability, 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1 ) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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